Pixel interpolation apparatus and method

ABSTRACT

In a pixel interpolation apparatus for interpolating pixels, an edge pixel detection unit detects edge pixels constituting an edge among pixels on lines positioned above/below the interpolation pixel. A continuous edge detection unit detects edge pixels in which two pixels or more consecutively line up among edge pixels detected by the edge pixel detection unit as a continuous edge. A continuous edge pair detection unit determines the combination of the continuous edges detected on each of the lines above and below the interpolation pixel among the continuous edges detected by the continuous edge detection unit. An edge direction determination unit determines the edge direction of the interpolation pixel on the basis of the positional relation of one set of continuous edges determined by the continuous edge pair detection unit. An interpolation pixel calculation unit calculates an interpolation pixel using the edge direction determined by the edge direction determination unit.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a technique for interpolating pixels ina spatial direction by interlace progressive transform, imageenlargement and the like.

2. Description of the Related Art

As examples of processes for interpolating pixels in the spatialdirection, there are interlace progressive transform (progressive scanconversion), image enlargement and the like. Of these, in the interlaceprogressive transform, generally an interpolation pixel value iscalculated by weighting and adding an intra-field interpolation pixelvalue obtained by intra-field interpolation and an inter-fieldinterpolation pixel value obtained by inter-field interpolationaccording to the motion of a pixel. Alternatively, either theintra-field interpolation pixel value or the inter-field interpolationpixel value is selected.

Since the pixels of each field exist only on every another line,generally bi-linear or bi-cubic interpolation in the vertical directionis performed using pixels on lines above or below a target interpolationpixel in the calculation of an intra-field interpolation pixel value. Ifpixel interpolation in the vertical direction is performed by theseinterpolation methods out of relation to the edge direction, aliasingcalled jaggies occur in inclined edge parts.

As a technique for suppressing the occurrence of jaggies by pixelinterpolation in inclined edge parts, for example, Japan PatentPublication No. 2005-341346 discloses a technique for determining whichthe surrounding area of an interpolation pixel is, a complex pattern, aflat pattern, a vertical-line pattern, an angular pattern or the like bydetecting a direction to interpolate using the differential value ofpixel information between adjacent pixels in the horizontal and verticaldirections in a field, and discriminating an inclined direction edge andthe like from the complex pattern and the like.

FIG. 1 shows the interpolation method of the Japan Patent PublicationNo. 2005-341346. Each point in FIG. 1 represents a pixel and of these,an edge pixel is represented by black color. If it is determined that itis the above patterns in the above determination, pixel interpolation isperformed in the vertical direction. If it is determined that it isother than the above patterns, pixel differences for each direction inthe surrounding area of the interpolation pixel shown In FIG. 1 iscalculated and pixel interpolation is performed determining a highlycorrelative direction as an interpolation direction, thereby to reducejaggies.

As another technique for reducing jaggies, for example, Japan PatentPublication No. 2005-266918 discloses a technique for determiningwhether there is an inclined direction edge about a focused pixel anddetecting the edge directions of a plurality of pixel positionscorresponding to the focused pixel position including the focused pixelon the basis of the determination result.

Japan Patent Publication No. 2005-341337 discloses a technique forcalculating the sum of the pixel information about an area around afocused pixel which is an interpolation target and its surrounding pixelareas respectively, detecting the highest correlative direction on thebasis of the pixel information of each of the calculated areas and thepositional relation between the areas and determining it as aninterpolation direction.

Furthermore, as another technique, Japan Patent Publication No.2007-67652 discloses a technique for determining an inclined directionof an edge in a position of a focused pixel, further determining a shapeof the edge in the position of the focused pixel and executingappropriate interpolation of the edge in the inclined directionaccording to the result of the determination of the shape of the edge.

Furthermore, as another technique, Japan Patent Publication No.2000-075865 discloses a technique for detecting an edge included in animage and calculating its direction, generating an adaptive samplingfunction by transforming a two-dimensional sampling function adapting itto the calculated edge direction and interpolating the edge part usingthe adaptive sampling function.

According to the technique disclosed by the Japan Patent Publication No.2005-341346, the limit of a gently inclined interpolation direction isrestricted by the width of a surrounding pixel area for determining theinterpolation direction. Specifically, if the width of the surroundingpixel area is small, the gently inclined interpolation direction cannotbe detected.

Conversely, if the width of a surrounding pixel area is large, even thegently inclined interpolation direction can be detected. However, if thewidth of a surrounding pixel area is large, it is often determined thatthe surrounding area is a complex area (pattern, etc.). Therefore, anedge direction for reducing jaggies is seldom interpolated, which is aproblem.

SUMMARY OF THE INVENTION

It is an object of the disclosed pixel interpolation apparatus andmethod to provide a technique for interpolating pixels in a correct edgedirection including a gently inclined edge direction in view of theabove problem.

In order to solve the problem, the disclosed pixel interpolationapparatus comprises an edge pixel detection unit for detecting edgepixels forming an edge, from pixels on lines positioned above/below orto the left/right of an interpolation pixel, a continuous edge detectionunit for detecting edge pixels in which two pixels or more consecutivelyline up among the edge pixels detected by the edge pixel detection unitas a continuous edge, a continuous edge pair detection unit forcombining continuous edges detected from lines above and below theinterpolation pixel or continuous edges detected from lines to the leftand right of the interpolation pixel among the continuous edges detectedby the continuous edge detection unit, an edge direction determinationunit for determining the edge direction of the interpolation pixel onthe basis of the positional relation between one set of continuous edgesdetermined by the continuous edge pair detection unit and aninterpolation pixel calculation unit for calculating an interpolationpixel using the edge direction determined by the edge directiondetermination unit.

When pixels detected as edges from pixels on lines above/below or to theleft/right of the interpolation pixel consecutively line up, thecontinuous pixels are detected as a continuous edge. When the continuousedge is on lines above/below or to the left/right of the interpolationpixel consecutively line up, two continuous edges are combined arespecified as one set of continuous edges (continuous edge pair). Then,an edge direction is determined using the continuous edge pair. Since aninterpolation pixel is calculated using an edge direction determinedthus, a correct edge direction can be calculated and correct pixelinterpolation can be performed even when the inclination of the edge isgentle.

The edge direction determination unit can also determine a directionobtained by connecting the middle point of each of the one set ofcontinuous edges as the edge direction of the interpolation pixel.Alternatively, the edge direction determination unit can determine adirection obtained by connecting the left or right end pixels of the oneset of continuous edges as the edge direction of the interpolationpixel.

The edge direction determination unit can also specify the edgedirection at the middle point of one continuous edge of the one set ofcontinuous edges as its initial value by the above edge directiondetermination method and consecutively determines an edge direction foreach interpolation pixel using the pixel correlation between thesurrounding pixels of the initial value of the edge direction of theinterpolation pixel from the middle point toward the outside of thecontinuous edge.

Alternatively, the edge direction determination unit can specify theedge direction at the end point of one continuous edge of the one set ofcontinuous edges as its initial value by the above edge directiondetermination method and consecutively determines an edge direction foreach interpolation pixel using the pixel correlation between thesurrounding pixels of the initial value of the edge direction of theinterpolation pixel from the end point towards another end point.

Furthermore, the edge direction determination unit can determines thespecified edge direction as the basic edge direction of the one set ofcontinuous edge pixels and consecutively determines a final edgedirection for each interpolation pixel on the basis of the basic edgedirection and the pixel correlation between the surrounding pixels ofthe basic edge direction of the interpolation pixel.

According to the disclosed pixel interpolation device, one set ofcontinuous edges can be detected from pixels adjacent the interpolationpixel, an edge direction can be calculated on the basis of the detectedone set of continuous edges and an interpolation pixel can becalculated. By calculating an edge direction from the combination ofactually detected continuous edges without limiting a range fordetermining an edge direction, even the edge direction of a gentlyinclined edge can be correctly determined, thereby correctly performingthe pixel interpolation of an edge direction.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows the conventional pixel interpolation method;

FIG. 2 shows the configuration of an interlace progressive transformdevice;

FIG. 3 shows the configuration of the pixel interpolation device;

FIG. 4 shows the configuration of the edge pixel detection unit;

FIG. 5 shows an example of an edge strength calculation filter;

FIG. 6A shows another example of an edge strength calculation filter;

FIG. 6B shows another example of an edge strength calculation filter;

FIG. 7 shows the continuous edge detection method;

FIG. 8 shows the detection method of a continuous edge pair (No. 1);

FIG. 9 also shows the detection method of a continuous edge pair (no.2);

FIG. 10 shows the configuration of the edge direction determinationunit;

FIG. 11A shows a method for determining an interpolation direction usinga continuous edge pair (No. 1);

FIG. 11B shows a method for determining an interpolation direction usinga continuous edge pair (No. 2);

FIG. 11C shows a method for determining an interpolation direction usinga continuous edge pair (No. 3);

FIG. 12 is a flowchart showing a process for determining an edgedirection;

FIG. 13 shows another configuration of the edge direction determinationunit;

FIG. 14 shows the calculation method of an edge direction-candidatecorrelation value;

FIG. 15A shows the determination method of an edge interpolationdirection (No. 1);

FIG. 15B shows the determination method of an edge interpolationdirection (No. 2);

FIG. 15C shows the determination method of an edge interpolationdirection (No. 3);

FIG. 16 shows a method for setting an interpolation direction candidatefor each target interpolation pixel (No. 1);

FIG. 17 shows a method for setting an interpolation direction candidatefor each target interpolation pixel (No. 2);

FIG. 18 is a flowchart showing a process for determining an edgedirection for each interpolation pixel (No. 1);

FIG. 19 is a flowchart showing a process for determining an edgedirection for each interpolation pixel (No. 2);

FIG. 20 shows one configuration of the interpolation pixel valuecalculation unit;

FIG. 21 shows the calculation method of an interpolation pixel value;

FIG. 22 shows another configuration of the interpolation pixel valuecalculation unit; and

FIG. 23 shows another calculation method of an interpolation pixelvalue.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The preferred embodiments of the present invention are described indetail below with reference to the drawings.

FIG. 2 shows the configuration of an interlace-progressive transformdevice. The interlace-progressive transform device 1 comprises aninter-field interpolation pixel value calculation unit 2, an intra-fieldinterpolation pixel value calculation unit 3, a motion detection unit 4and a pixel value blend unit 5.

Although a pixel interpolation method according to this preferredembodiment can also be applied to image enlargement, in the followingdescription, pixel interpolation in interlace progressive transform isdescribed as an example.

The inter-field interpolation pixel value calculation unit 2 detects themotion between two images in the field of the same parity, of inputtedimage signals, and calculates the interpolation pixel value of a fieldimage of a different parity (this is called interpolation-target fieldimage) between the two fields where the motion is detected, using thepixel of a field image at a time different from that of theinterpolation-target field image. The intra-field interpolation pixelvalue calculation unit 3 applies bi-linear or bi-cubic interpolation andthe like to the interpolation-target field image using pixels in thefield.

The motion detection unit 4 calculates motion vectors from two images inthe field of the same parity and detects their motion. The pixel valueblend unit 5 weights the inter-field interpolation pixel value and theintra-field interpolation pixel value using the result of the motiondetected by the motion detection unit 4, and generates and outputs anoutput image.

A pixel interpolation method according to this preferred embodiment isimplemented in the intra-field interpolation pixel value calculationunit 3 of the interlace-progressive transform device 1 shown in FIG. 2.The intra-field interpolation pixel value calculation unit, that is,pixel interpolation device 3 is described below.

FIG. 3 shows the configuration of the pixel interpolation device(intra-field interpolation pixel value calculation unit) 3. The pixelinterpolation device 3 shown in FIG. 3 comprises an edge pixel detectionunit 11, a continuous edge detection unit 12, a continuous edge pairdetection unit 13, an edge direction determination unit 14 and aninterpolation pixel value calculation unit 15.

The edge pixel detection unit 11 detects edge pixels from linesabove/below the interpolation pixel. The continuous edge detection unit12 detects the edge pixels that consecutively line up from the edgepixels detected by the edge pixel detection unit 11 as a continuous edgewhen some edge pixels line up in a certain line.

The continuous edge pair detection unit 13 detects continuous edges asone set of continuous edges when there are continuous edges on linesabove/below the interpolation pixel among the edges detected by thecontinuous edge detection unit 12. Hereinafter, one set of continuousedges detected by the continuous edge pair detection unit 13 is definedas “continuous edge pair”.

The edge direction determination unit 14 determines the interpolationdirection of the continuous edge pair detected by the continuous edgepair detection unit 13. The interpolation pixel value calculation unit15 calculates an interpolation pixel value using the interpolationdirection determined by the edge direction determination unit 14.

As described above, the pixel interpolation device 3 shown in FIG. 3detects one set of edge pixels positioned on lines above/below theinterpolation pixel, of edge pixels that consecutively line up as acontinuous edge pair and determines an interpolation direction on thebasis of the continuous edge pair. Since an interpolation direction isdetermined using an actually detected continuous edge pair withoutlimiting it to within the range of a prescribed area as in the priorart, the inclination of an edge is not limited and even a gentlyinclined edge direction can also be correctly detected.

In this preferred embodiment, since pixel interpolation ininterlace-progressive transform is described, edge pixels and continuousedges are detected from lines above/below the interpolation pixel. Forexample, in a process, such as image enlargement and the like, edgepixels and continuous edges are detected from not only lines above/belowthe interpolation pixel but also lines to its left/right.

The configuration and its operation of each unit of the pixelinterpolation device 3 shown in FIG. 3 are described in detail below.

FIG. 4 shows the configuration of the edge pixel detection unit 11. Theedge pixel detection unit 11 shown in FIG. 4 comprises an edge intensitycalculation unit 21, an edge direction calculation unit 22 and an edgepixel determination unit 23.

The edge intensity calculation unit 21 calculates edge intensity usingan edge strength calculation filter.

FIG. 5 shows an example of an edge intensity calculation filter. Theedge intensity calculation filter calculates edge intensity bycalculating a brightness correlation value between a focused pixel andits peripheral pixel. As the edge intensity calculation filter is awell-known technique, its detailed description is omitted here.

FIGS. 6A and 6B show other examples of the edge intensity calculationfilter. FIG. 6A is an example of the edge intensity calculation filterfor the vertical direction. FIG. 6B is an example of the edge intensitycalculation filter for the horizontal direction. Edge intensity in boththe vertical and horizontal directions can also be separately calculatedusing the filters shown in FIGS. 6A and 6B.

The edge direction calculation unit 22 calculates an edge angle (edgedirection) from the ratio between edge intensity in the vertical andhorizontal directions.

The edge pixel determination unit 23 determines an edge pixel using theedge intensity and direction calculated by the edge intensitycalculation unit 21 and the edge direction calculation unit 22,respectively. More specifically, a pixel whose edge intensity is equalto or more than a predetermined threshold and whose edge direction isalso equal to or less than a predetermined angle is detected as an edgepixel.

FIG. 7 shows the continuous edge detection method. Each point shown inFIG. 7 represents a pixel and of these points, a pixel detected as anedge pixel by the edge pixel detection unit 11 is represented by blackcolor.

As shown in FIG. 7, when a plurality of edge pixels, consecutivelylining up, of the detected edge pixels is represented by a certain line,those edge pixels are detected as a continuous edge. The fact that apredetermined number or more of edge pixels consecutively line up canalso be made an additional condition.

FIG. 8 shows a detection method of a continuous edge pair. Of theexisting lines shown in FIG. 8, a continuous edge is represented by asolid line. FIG. 8 shows a case where three continuous edges exist onthe existing lines above/below the interpolation line.

In the preferred embodiment, when there is one pixel or more overlaps inthe continuous edges on lines above/below the interpolation line themethod for detecting a continuous edge pair from continuous edgesdetected from the existing lines positioned above/below theinterpolation line detects the combination of the continuous edges as acontinuous edge pair.

The fact that a ratio in length between the continuous edges is equal toor less than a predetermined threshold can also be an additionaldetection condition. Furthermore, the detection condition shown in FIG.9 can also be added.

FIG. 9 shows another detection method of the continuous edge pair. Whenthere is one set of continuous edges their parts of which overlap eachother, a brightness difference between a pixel on the existing lineincluding the continuous edges and a pixel on another existing line inthe left/right areas divided by the one set of continuous edges iscalculated. The fact the calculated brightness difference is equal to orless than a predetermined threshold in each of the left/right areas canbe made an additional detection condition. This additional detectioncondition is based on the assumption that the brightness values arewithin a certain range in the left or right side area of the edge.

FIG. 10 shows a configuration of the edge direction determination unit14. The edge direction determination unit 14 shown in FIG. 10 comprisesan edge direction calculation unit 41.

The edge direction calculation unit 41 determines the edge direction andpixel interpolation area of the interpolation pixel on the basis of thepositional relation between continuous edges forming the continuous edgepair detected by the continuous edge pair detection unit 13. Morespecifically, the edge direction and pixel interpolation area for eachinterpolation pixel are calculated on the basis of the positionalinformation of the inputted continuous edge pair and the brightnessvalues of the continuous edge pair and its peripheral pixels.

The positional information of the continuous edge pair comprisesinformation indicating the position of the starting pixel of eachcontinuous edge and information indicating the position of its endingpixel.

FIGS. 11A, 11B and 11C show the method for determining an interpolationdirection using a continuous edge pair. As shown in FIG. 11A, forexample, a direction obtained by connecting the middle point of eachcontinuous edges of the continuous edge pair can be made an edgedirection. Alternatively, as shown in FIGS. 11B and 11C, a directionobtained by connecting the left or right ends of each continuous edgesof the continuous edge pair can be made an edge direction.

FIG. 12 is a flowchart showing a process for determining an edgedirection using the above method.

The process shown in FIG. 12 is started with the input to the edgedirection determination unit 14 of the positional information of thecontinuous edge pair and the brightness values of the continuous edgepair and its peripheral pixel as a trigger.

Firstly, in step S1 an edge direction and a pixel interpolation area aredetermined for each continuous edge pair. As described above, the edgedirection can be obtained by calculating the position of the middlepoint, left end or right end from the positional information of aninputted continuous edge pair. The pixel interpolation area can beobtained by calculating an area to interpolate on the basis of thebrightness values of an inputted continuous edge pair and its peripheralpixel.

In step S2, the edge direction of the continuous edge pair is set as theedge direction of the interpolation pixel of the pixel interpolationarea calculated in step S1 and the process is terminated.

FIG. 13 shows another configuration of the edge direction determinationunit. The edge direction determination unit 14B shown in FIG. 13comprises an edge direction initial value determination unit 42, an edgedirection-candidate correlation value calculation unit 43 and a finaledge direction determination unit 44. In the description of FIGS. 13 andafter, a reference numeral “14B” is attached to this edge directiondetermination unit in order to discriminate it from the edge directiondetermination unit 14 shown in FIG. 10.

The edge direction determination unit 14B shown in FIG. 13 differs fromthe edge direction determination unit 14 shown in FIG. 10 in determiningthe edge direction calculated by the same method as described above asthe initial value of the edge direction and determining a final edgedirection on the basis of the calculated edge direction initial valueand the correlation between the interpolation pixel and its peripheralpixel.

The edge direction initial value determination unit 42 calculates anedge direction by the same method as in the edge direction calculationunit 41 shown in FIG. 10 and specifies it as the initial value of theedge direction. The edge direction initial value determination unit 42outputted the brightness values of the continuous edge pair and itsperipheral pixel in addition to the edge direction and pixelinterpolation area of each continuous edge pair.

The edge direction-candidate correlation value calculation unit 43calculates the pixel correlation value of the edge direction candidateof the peripheral pixel of the interpolation pixel using the informationinputted from the edge direction initial value determination unit 42,that is, the edge direction of each continuous edge pair, the pixelinterpolation area and the brightness values of the continuous edge pairand its peripheral pixel. Then, the edge direction-candidate correlationvalue calculation unit 43 outputs the pixel correlation value and pixelinterpolation area of each interpolation pixel. The detailed calculationmethod of the pixel correlation value will be described later.

The final edge direction determination unit 44 determines a final edgedirection for each interpolation pixel on the basis of the informationinputted from the edge direction-candidate correlation value calculationunit 43, that is, the pixel correlation value and pixel interpolationarea of the edge direction candidate of each interpolation pixel andoutputs the edge direction and pixel interpolation area for each pixel.

FIG. 14 shows the calculation method of an edge direction-candidatecorrelation value. Of the pixels shown in FIG. 14, pixels A to E andpixels F to J represent pixels on the existing lines positionedabove/below the interpolation line, respectively. A case where theinitial value of an edge direction is pixels C-H is described below.

The two pixels to the left/right of each pixel of the initial values C-Hof the edge direction are extracted and their correlation values arecalculated. Specifically, |C-H|, |A-J|, |B-I|, |D-G| and |E-F| arecalculated. For the detailed calculation method of the correlationvalue, a well-known technique is used.

For the calculation method of the correlation value other than theabove-described, for example, pixels other than the pixels in the edgedirection (pixels C and H in FIG. 14) can also be used. For example, theaverage of a predetermined number of pixels in the horizontal direction(for example, five pixels) using the pixel in the edge direction as thecenter can also be used.

FIGS. 15A, 15B and 15C show the determination method of an edgeinterpolation direction.

As shown in FIG. 15A, the initial value of the edge direction can alsobe calculated using a pixel positioned at the middle point of eachcontinuous edge and an edge interpolation direction can also beconsecutively calculated from the middle point of each continuous edgesforming the continuous edge pair toward the outside.

Alternatively, as shown in FIGS. 15B and 15C, the initial value of theedge direction can be calculated using pixels positioned at theleft/right ends of each continuous edge and an edge interpolationdirection can be consecutively calculated toward the right and leftsides, respectively.

When any of pixels above/below the interpolation pixel goes beyond thearea of the continuous edge, interpolation can also be terminated. Orelse interpolation can be executed outside the continuous edge. That is,interpolation can be terminated when both of pixels above/below theinterpolation pixel goes beyond the area of the continuous edge.

FIGS. 16 and 17 show a method for setting an interpolation directioncandidate for each target interpolation pixel.

For example, as shown in FIG. 16, when determining an interpolationdirection using D-M as the initial values and then determining theinterpolation direction of a target interpolation pixel positioned tothe right by one pixel, the correlation value can also be calculated bythe above method using E-N obtained by shifting each of the calculatedinitial values D-M to the right by one pixel as the center.

Alternatively, as shown in FIG. 17, when determining the E-L as theinterpolation direction using D-M as the initial values and thendetermining the interpolation direction of a target interpolation pixelpositioned to the right one pixel, an correlation value can becalculated by the above method, using a direction F-M obtained byconnecting pixels F and M shifted each of the pixels E and L in thedetermined interpolation direction to the right by one pixel as thecenter.

FIG. 18 is a flowchart showing a process for setting an interpolationdirection candidate by the method shown in FIG. 16 and determining anedge direction for each interpolation pixel. As described above,according to the method shown in FIG. 16, the correlation value of aninterpolation direction candidate to subsequently determine can becalculated using a direction obtained by shifting pixels in thecontinuous edge to a prescribed direction (to the left or right) by onepixel in parallel with the interpolation direction specified as theinitial value.

Like the process shown in FIG. 12, the process shown in FIG. 18 isstarted with the input to the edge direction determination unit 14B ofthe positional information of the continuous edge pair and thebrightness values of the continuous edge pair and its peripheral pixelas a trigger. As described above, the positional information of thecontinuous edge pair comprises information indicating the position ofthe starting pixel of each continuous edge pair and informationindicating the position of its ending pixel.

Firstly, in step S11, the same process as in step S1 of FIG. 12 isperformed and an edge direction and a pixel interpolation area aredetermined for each continuous edge pair. In step S12 the determinededge direction of each continuous edge pair is set as the initial valueof the edge direction candidate of an interpolation pixel in the pixelinterpolation area.

In step S13, as to an interpolation target pixel, the pixel correlativevalue of the edge direction candidate is calculated using the edgedirection initial value as the center and in step S14, as to theinterpolation target pixel, an edge direction is determined of the edgedirection candidates, using the pixel correlative value. Then, in stepS15, it is determined whether the interpolation of all the pixels in thepixel interpolation area of the continuous edge pair is completed.Although all the pixels in each continuous edge pair forming thecontinuous edge pair are interpolated in FIG. 18, the interpolation isnot limited to this. As described earlier, interpolation can also becarried out until up to the end of one continuous edge is completed.

If in step S15 it is determined that the interpolation of all the pixelsin the pixel interpolation area of the continuous edge pair is notcompleted yet, the flow returns to step S13 and the same process isapplied to pixels in one predetermined direction of the right or leftside by one pixel.

After this, the processes in step S13 and S14 are repeatedly applied toeach interpolation pixel and if it is determined that the interpolationof all the pixels in the pixel interpolation area of the continuous edgepair is completed in step S15, the process is terminated.

FIG. 19 is a flowchart showing a process for setting interpolationdirection candidates by the method shown in FIG. 17 and determining anedge direction for each interpolation pixel. As described above,according the method shown in FIG. 17, a correlative value is calculatedaround a direction obtained by shifting the interpolation directiondetermined of a specific interpolation target pixel to the predetermineddirection (the right or left side) by one pixel in parallel.

The process shown in FIG. 19 is started using the input to the edgedirection determination unit 14B, of the positional information of acontinuous edge pair and the brightness values of the continuous edgepair and its peripheral pixel as a trigger, as in the processes shown inFIGS. 12 and 18. The positional information of a continuous edge pair iscomposed of information indicating the position of the start pixel ineach continuous edge and information indicating the position of the endpixel.

Firstly, in step S21 the same process as in step S1 of FIG. 12 and stepS11 of FIG. 18 is performed. In step S22 the edge direction of thecontinuous edge pair is set as the initial value of the edge directioncandidate of an interpolation pixel for determining an edge direction inthe pixel interpolation area first.

The processes in steps S23 and S24 correspond to those of steps S13 andS14, respectively, of FIG. 18.

In step S25 the edge direction determined in step S24 is set as theinitial value of the edge direction candidate of a subsequentinterpolation pixel. Then, in step S26, as in step S15 of FIG. 18, it isdetermined whether the interpolation of all pixels in the pixelinterpolation area of the continuous edge pair. If the interpolation isnot completed, the flow returns to step S23 and the same process isrepeated. If it is determined that the interpolation of all the pixelsis completed in step S26, the process is terminated.

FIG. 20 shows a configuration of the interpolation pixel valuecalculation unit 15. The interpolation pixel value calculation unit 15shown in FIG. 20 comprises an edge direction interpolation valuecalculation unit 51.

The edge direction interpolation value calculation unit 51 calculates aninterpolation pixel value on the basis of an image inputted to the pixelinterpolation device 3 and the edge direction for each pixel, determinedby the edge direction determination unit 14 and outputs it.

FIG. 21 shows one calculation method of an interpolation pixel value. InFIG. 21 pixels on three lines are represented by a circle. Of these,pixels on the top and bottom lines represent pixels on the existing lineand pixels in the middle line represent pixels on the interpolationline.

The interpolation pixel value n of an interpolation pixel P iscalculated as follows using the pixel values α and β of a pixelpositioned in the edge direction of the interpolation pixel P of thepixel value calculation target of pixels on the actually existing topand bottom lines.

η=(α+β)/2   (1)

FIG. 22 shows another configuration of the interpolation pixel valuecalculation unit. The interpolation pixel value calculation unit 15Bshown in FIG. 22 comprises an edge direction interpolation pixel valuecalculation unit 51, a vertical-direction interpolation pixelcalculation unit 52, an edge direction interpolation pixelvalue/vertical-direction interpolation pixel value blend ratiocalculation unit (hereinafter called blend ratio calculation unit) 53and an edge direction interpolation pixel value/vertical-directioninterpolation pixel value blend unit (hereinafter called blend unit) 54.In the description of FIG. 22 and after, “15B” is attached to thisinterpolation pixel value calculation unit in order to discriminate itfrom the interpolation pixel value calculation unit 15 shown in FIG. 20.

Since this edge direction interpolation pixel value calculation unit 51is the same as that shown in FIG. 20, its description is omitted here.

The vertical-direction interpolation pixel calculation unit 52calculates an interpolation pixel value on the basis of pixelspositioned above/below the interpolation line in the vertical directionof the interpolation pixel. The blend ratio calculation unit 53calculates a blend ratio between the interpolation pixel value(hereinafter called edge direction interpolation pixel value) calculatedby the edge direction interpolation pixel value calculation unit 51 andthe interpolation pixel value (hereinafter called vertical directioninterpolation pixel value) calculated by the vertical directioninterpolation pixel value calculation unit 52.

The blend unit 54 calculates a final interpolation pixel value on thebasis of the edge direction interpolation pixel value and the verticaldirection interpolation pixel value, using the blend ratio calculated bythe blend ratio calculation unit 53 and outputs it.

FIG. 23 shows the calculation method of an interpolation pixel value inthe interpolation pixel value calculation unit 15B shown in FIG. 22. InFIG. 23, as in FIG. 21, pixels on the existing line are shown in the topand bottom line, and pixels on the interpolation line are shown in themiddle line.

The calculation method of the interpolation pixel value η in the edgedirection of the interpolation pixel P is as already described withreference to FIG. 21.

The vertical direction interpolation pixel value γ of the interpolationpixel P can be calculated as follows using the pixel values A and B ofpixels positioned in the vertical position of the interpolation pixel P,of pixels on the top and bottom existing lines shown in FIG. 23.

γ=(A+B)/2   (2)

The calculation method of a vertical direction interpolation pixel valueis not limited to the above-described and can also be calculated by thebi-cubic interpolation in the vertical direction.

The weight value ε of the edge direction interpolation and the weightvalue κ of the vertical direction interpolation pixel value used in theblend ratio between the edge direction pixel value η and the verticalinterpolation pixel value γ are calculation as follows.

ε=|A−B|  (3)

κ=|α−β|  (4)

The final interpolation pixel value π of the interpolation pixel P canbe calculated as follows, for example, using η, γ, ε and κ calculated bythe above equations (1) to (4).

π=(ε*η+κ*γ)/(ε+κ)   (5)

In this case, it is assumed that if ε=κ=0, π=γ.

The calculation method of the weighting value ε of the edge directioninterpolation pixel value is not limited to the above equation (3). Forexample, the average of the differential absolute values of pixels onlines above/below the interpolation pixel can also be calculated usingnot only pixels above/below the interpolation pixel but also severalpixels (for example, seven pixels) in its horizontal direction aroundpixels above/below the interpolation pixel.

As described above, according to a pixel interpolation device accordingto this preferred embodiment, when continuous edges formed by continuousedge pixels are positioned on lines above/below (or to the left/rightof) the interpolation line, these continuous edges can be detected as acontinuous edge pair, an edge direction can be calculated on the basisof the detected continuous edge pair and an interpolation pixel valuecan be calculated. By calculating an edge direction on the basis of thecombination of continuous edges actually detected as edges instead ofcalculating an edge direction in a prescribed range, as to even a gentlyinclined edge, an edge direction can be correctly determined.Furthermore, thus, jaggies can be reduced by correctly interpolatingpixels.

1. A pixel interpolation apparatus for interpolating pixels, comprising:an edge pixel detection unit for detecting edge pixels forming an edge,from pixels on lines positioned above/below or to the left/right of aninterpolation pixel; a continuous edge detection unit for detecting edgepixels in which two pixels or more consecutively line up among the edgepixels detected by the edge pixel detection unit as a continuous edge; acontinuous edge pair detection unit for combining continuous edgesdetected from lines above and below the interpolation pixel orcontinuous edges detected from lines to the left and right of theinterpolation pixel among the continuous edges detected by thecontinuous edge detection unit; an edge direction determination unit fordetermining an edge direction of the interpolation pixel on the basis ofpositional relation between one set of continuous edges specified by thecontinuous edge pair detection unit; and an interpolation pixelcalculation unit for calculating an interpolation pixel using the edgedirection determined by the edge direction determination unit.
 2. Thepixel interpolation apparatus according to claim 1, wherein the edgepixel detection unit calculates an edge strength and an edge directionof a pixel and detects an edge pixel using the calculated edge strengthand edge direction.
 3. The pixel interpolation apparatus according toclaim 1, wherein the edge direction determination unit determines adirection obtained by connecting a middle point of each of the one setof continuous edges as an edge direction of the interpolation pixel. 4.The pixel interpolation apparatus according to claim 1, wherein the edgedirection determination unit determines a direction obtained byconnecting pixels at the left or right end of each of the one set ofcontinuous edges as an edge direction of the interpolation pixel.
 5. Thepixel interpolation apparatus according to claim 1, wherein the edgedirection determination unit determines an edge direction at a middlepoint of one continuous edge of the one set of continuous edges by theedge direction determination method, as an initial value of the edgedirection of an interpolation pixel, and consecutively determines anedge direction for each interpolation pixel from the middle point towardoutside the continuous edge using pixel correlation of peripheral pixelsof the initial value of the edge direction of an interpolation pixel. 6.The pixel interpolation apparatus according to claim 1, wherein the edgedirection determination unit determines an edge direction at an endpoint of one continuous edge of the one set of continuous edges by theabove edge direction determination method, as an initial value of theedge direction of an interpolation pixel, and consecutively determinesan edge direction for each interpolation pixel from the endpoint towardanother endpoint using pixel correlation of peripheral pixels of theinitial value of the edge direction of an interpolation pixel.
 7. Thepixel interpolation apparatus according to claim 1, wherein the edgedirection determination unit specifies a specified edge direction as abasic edge direction of the one set of continuous edge pixels andconsecutively determines a final edge direction on the basis of thebasic edge direction and pixel correlation of peripheral pixels of thebasic edge direction of an interpolation pixel.
 8. The pixelinterpolation apparatus according to claim 1, wherein the interpolationpixel calculation unit calculates an interpolation pixel value usingpixels in an edge direction determined by the edge directiondetermination unit.
 9. The pixel interpolation apparatus according toclaim 1, wherein the interpolation pixel calculation unit calculates aninterpolation pixel value of en edge direction using pixels in the edgedirection determined by the edge direction determination unit, and alsocalculates an interpolation pixel value of a vertical direction usingpixels in the vertical direction of the one set of continuous edges,weights the interpolation pixel value in the edge direction and theinterpolation pixel value in the vertical direction and outputs aninterpolation pixel.
 10. A pixel interpolation method, comprising:detecting edge pixels forming an edge, from pixels on lines positionedabove/below or to the left/right of an interpolation pixel; detectingedge pixels in which two pixels or more consecutively line up among thedetected edge pixels as a continuous edge; determining a combination ofcontinuous edges detected from lines above and below the interpolationpixel or continuous edges detected from lines to the left and right ofthe interpolation pixel among the detected continuous edges; determiningan edge direction of the interpolation pixel on the basis of positionalrelation between the specified one set of continuous edges; andcalculating an interpolation pixel using the determined edge.